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INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-4704 


March  20,  2001 


MEMORANDUM  FOR  NAVAL  INSPECTOR  GENERAL 

SUBJECT :  Audit  Report  on  the  On-Board  Jammers  for  the  Integrated  Defensive 
Electronic  Countermeasures  (Report  No.  D-200 1-086) 


We  are  providing  this  report  for  review  and  comment.  We  considered 
management  comments  on  a  draft  of  this  report  when  preparing  the  final  report. 

DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 
The  Deputy  Assistant  Secretary  of  the  Navy  for  Air  Programs  did  not  concur  with 
Recommendation  2.  Therefore,  we  request  that  the  Deputy  Assistant  Secretary  of  the 
Navy  for  Air  Programs  provide  additional  comments  on  Recommendation  2.  by 
May  21,  2001. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  For  additional 
information  on  this  report,  please  contact  Mr.  Charles  M.  Santoni  at  (703)  604-9051 
(DSN  664-9051)  (csantoni@dodig.osd.mil)  or  Mr.  Robert  L.  Shaffer  at  (703)  604-9043 
(DSN  664-9043)  (rshaffer@dodig.osd.mil).  See  Appendix  C  for  the  report  distribution. 
The  audit  team  members  are  listed  inside  the  back  cover. 


Thomas  F.  Gimble 
Acting 

Deputy  Assistant  Inspector  General 
for  Auditing 
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On-Board  Jammers  for  the  Integrated  Defensive 
Electronic  Countermeasures 


Executive  Summary 


Introduction.  The  Integrated  Defensive  Electronic  Countermeasures  suite  is  intended 
to  provide  self-protection  and  increased  survivability  for  tactical  aircraft  against  radio 
frequency  and  infrared  surface-to-air  and  air-to-air  threats.  The  Navy-led  program  is 
designed  to  fill  the  electronic  self-protection  operational  deficiency  for  the  Navy 
F/A-18  E/F  aircraft  and  will  be  integrated  on  the  Air  Force’s  B-1B  and  F-15  aircraft. 
The  major  hardware  components  to  be  developed  are  the  AN/AFQ-214  radio  frequency 
countermeasures  system  and  the  AN/AFE-55  Fiber  Optics  Towed  Decoy.  Total 
program  cost  is  estimated  to  be  $2.7  billion  (then-year  dollars).  The  AN/AFQ-214 
development  cost  is  $85.8  million  and  the  procurement  cost  is  $1.1  billion  (458  units  at 
$2.3  million  each).  The  Navy  plans  to  have  an  inventory  of  366  radio  frequency 
countermeasures  systems  and  10,980  decoys.  The  Air  Force  plans  to  have  an  inventory 
of  92  radio  frequency  countermeasures  systems,  3,069  decoys  for  the  B-1B,  and 
15,048  decoys  for  the  F-15. 

Objectives.  The  overall  objective  was  to  evaluate  the  test  planning  and  requirements 
for  the  AN/AFQ-165  and  AN/AFQ-214  on-board  jammers.  We  also  reviewed  the 
management  control  program  as  it  applied  to  our  audit  objective. 

Results.  The  Navy  significantly  reduced  mission  reliability  from  the  level 
recommended  in  the  cost  and  operational  effectiveness  analysis.  The  Navy  reduced  the 
requirements  so  that  the  AN/AFQ-165  Airborne  Self-Protection  Jammer,  which  the 
Navy  plans  to  use  in  Block  I  of  the  Integrated  Defensive  Electronic  Countermeasures 
Suite,  could  pass  the  operational  test  and  evaluation  and  be  installed  on  the  F/A-18  E/F 
aircraft.  Furthermore,  the  AN/AFQ-214,  which  will  be  the  on-board  jammer  for 
Blocks  II  and  III  of  the  Integrated  Defensive  Electronic  Countermeasures  Suite  will  be 
tested  against  the  same  operational  suitability  requirements.  By  reducing  the  mission 
reliability  rate,  the  Navy’s  logistical  support  requirement  may  have  to  be  significantly 
increased  in  order  to  accomplish  a  90  percent  operational  availability  rate  for  the 
system.  At  the  reduced  mission  reliability  rate,  unscheduled  maintenance  may  be 
required  up  to  2.5  times  more  often  than  if  the  system  met  the  mission  reliability  rate 
recommended  by  the  cost  and  operational  effectiveness  analysis.  Additionally,  it  is 
unclear  whether  the  additional  protection  provided  by  the  on-board  jamming  capability 
justifies  the  investment  in  the  development,  acquisition  and  logistical  support.  For 
details  of  the  audit  results,  see  the  Finding  section  of  this  report.  Management  controls 
were  adequate  in  that  we  did  not  identify  any  systemic  management  control  weakness 
applicable  to  our  audit  objective.  See  Appendix  A  for  details  on  the  management 
control  program. 


Summary  of  Recommendations.  We  recommend  that  the  Program  Manager, 
Integrated  Defensive  Electronic  Countermeasures,  prior  to  proceeding  with  Block  II  of 
the  Program,  determine  the  logistics  support  cost  to  maintain  the  on-board  jamming 
capability  based  on  the  results  of  the  operational  test  and  evaluation.  We  also 
recommend  that  the  Program  Manager  reassess  whether  the  on-board  jamming 
capability  provides  an  amount  of  added  protection  that  justifies  its  development, 
acquisition,  and  logistical  support  costs. 

Navy  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy  for  Air  Programs  (the 
Deputy  Assistant  Secretary)  nonconcurred  with  the  finding,  stating  that  the  mission 
reliability  requirement  was  based  upon  a  systems  engineering  analysis;  that  current 
logistics  cost  requirements  were  based  upon  the  system  (including  on-board  jammers) 
as  it  was  intended  to  be  fielded;  and  that,  as  non-developmental  items,  the  on-board 
jammers  have  minimal  impact  in  terms  of  development,  acquisition  and  logistics  costs. 

The  Deputy  Assistant  Secretary  agreed  to  review  the  logistics  support  costs  in 
preparation  for  the  Milestone  III  decision  but  did  not  agree  to  reassess  whether  the 
added  protection  provided  by  having  on-board  jamming  capability  justifies  its 
development,  acquisition  and  logistical  support  costs.  The  Assistant  Secretary  stated 
that  cost  versus  benefit  of  on-board  jamming  capability  was  assessed  by  the  Johns 
Hopkins  University  Applied  Physics  Laboratory  in  May  1999  and  that  another 
reassessment  is  not  necessary.  A  discussion  of  the  management  comments  is  in  the 
Finding  section  of  this  report,  and  the  complete  text  is  in  the  Management  Comments 
section. 

Audit  Response.  On  January  19,  1995,  the  Johns  Hopkins  University  Applied  Physics 
Laboratory  issued  a  cost  and  operational  effectiveness  analysis  on  alternative  electronic 
warfare  suites.  The  analysis  recommended  that  the  preferred  alternative,  regardless  of 
whether  that  system  utilized  an  on-board  jammer,  have  a  mission  reliability  requirement 
of  94  percent.  The  system  being  developed  by  the  Navy  does  not  meet  that 
requirement.  Furthermore,  discussions  with  officials  of  the  Office  of  the  Deputy  Chief 
of  Naval  Operations  (Resources,  Requirements,  and  Assessments)  indicated  that  the 
Navy  was  aware  that  Block  I  of  the  Integrated  Defensive  Electronic  Countermeasure 
(the  Airborne  Self-Protection  Jammer  and  the  existing  AN /ALE-50  Towed  Decoy) 
would  be  destined  for  failure  if  the  system  was  required  to  meet  a  94  percent  mission 
reliability.  We  agree  that  using  Airborne  Self-Protection  Jammer  as  the  on-board 
jammer  for  Block  I  of  the  Integrated  Defensive  Electronic  Countermeasure  will  have 
minimal  impact  in  terms  of  development  and  acquisition  costs.  However,  we  disagree 
that  development  and  procurement  costs  of  AN/ALQ-214  for  Blocks  II  and  III  of  the 
Integrated  Defensive  Electronic  Countermeasure  are  insignificant.  The  AN/ALQ-214 
development  cost  is  $85.8  million  and  the  procurement  cost  is  $1.1  billion  (458  units  at 
$2.3  million  each).  This  represents  about  44  percent  of  the  $2.7  billion  total  program 
costs  for  the  integrated  defensive  electronic  countermeasure. 

We  consider  the  Navy  statement  that  it  will  review  the  logistics  support  costs  before  the 
Milestone  III  decision  to  be  responsive  to  the  recommendation.  The  May  1999  analysis 
referenced  by  the  Navy  determined  that,  although  the  use  of  the  towed  decoy  was  the 
most  effective  alternative  against  threats,  consideration  should  be  given  to  using  on¬ 
board  transmitters  in  conjunction  with  the  towed  decoy  system  as  part  of  the  F/A-18 
electronic  warfare  suite.  However,  the  analysis  did  not  address  the  cost  versus  the 
additional  benefit  that  on-board  transmitters  would  provide.  Therefore,  we  request  that 
the  Deputy  Assistant  Secretary  of  the  Navy  for  Air  Programs  provide  additional 
comments  by  May  21,  2001. 
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Background 


In  May  1992,  the  Director,  Operational  Test  and  Evaluation,  assessed  that  the 
AN/ALQ-165  Airborne  Self-Protection  Jammer  (ASPJ)  was  not  operationally 
effective  during  operational  test  and  evaluation.  The  Director,  Operational  Test 
and  Evaluation,  also  assessed  ASPJ  as  not  operationally  suitable  because  it  did 
not  meet  required  criteria  for  mission  reliability  or  built-in-test  effectiveness. 
Mission  reliability  is  the  probability  that  a  system  will  perform  mission-essential 
functions  for  a  period  of  time  under  conditions  stated  in  the  mission  profile. 
Built-in-test  effectiveness  is  the  probability  that  built-in-tests  will  correctly  detect 
a  fault,  which  minimizes  the  removal  of  components  because  of  incorrect 
indications  from  the  built-in-tests.  On  December  15,  1992,  DoD  canceled  the 
ASPJ  program  and  production  was  terminated. 

After  cancellation  of  the  ASPJ  program,  the  Navy  conducted  analyses  that 
generated  a  requirement  for  a  tactical  aircraft  self-protection  capability  against 
radio  frequency  and  infrared  surface-to-air  and  air-to-air  threats.  The  Integrated 
Defensive  Electronic  Countermeasures  (IDECM)  Mission  Need  Statement 
discusses  deficiencies  in  five  areas: 

•  self-contained  radio  frequency  self-protection, 

•  missile  approach  warning, 

•  infrared  and  ultraviolet  expendables, 

•  integration  of  the  subsystems  of  the  self-protection  suite,  and 

•  identification  of  precise  direction  of  radio  frequency  transmitters. 

Development  of  the  IDECM  suite  began  in  FY  1996  as  an  Acquisition 
Category  II  program.  The  IDECM  program  began  as  only  an  off-board 
jammer,  specifically  the  Fiber  Optic  Towed  Decoy,  with  a  complementary  on¬ 
board  radio  frequency  jammer  capability  as  a  possible  upgrade.  The  IDECM 
suite  was  intended  to  provide  greater  self-protection  and  increased  survivability 
against  a  wider  range  of  threats  than  an  aircraft  without  self-protection  or  was 
equipped  with  existing  countermeasures.  The  IDECM  suite  is  a  Navy-led 
program  designed  for  the  F/A-18  E/F  aircraft  to  resolve  the  electronic  self¬ 
protection  operational  deficiency.  Portions  of  the  suite  will  also  be  integrated 
on  the  Air  Force’s  B-1B  and  F-15  aircraft.  The  major  hardware  components  to 
be  developed  are  the  IDECM  radio  frequency  countermeasures  system 
(AN/ALQ-214)  and  the  AN /ALE-55  Fiber  Optics  Towed  Decoy. 

Cost  and  technical  issues  have  caused  schedule  delays  in  the  IDECM  program. 
To  ensure  that  the  initial  deployments  of  the  F/A-18  E/F  aircraft  have  self¬ 
protection,  the  Navy  decided  to  include  an  on-board  radio  frequency  jammer  in 
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the  suite.  The  Director,  Air  Warfare  Division,  Office  of  the  Deputy  Chief  of 
Naval  Operations  (Resources,  Requirements  and  Assessments),  added  the 
requirement  for  on-board  jammers  to  provide  protection  in  the  event  that  the 
three  available  towed  decoys  were  expended  or  failed  to  operate.  To  reduce 
risk  and  to  provide  the  electronic  warfare  protection,  the  Navy  established  a 
phased  approach.  Three  sequential  phases  of  on-board  jammers  and  towed 
decoys  were  to  be  used. 

•  IDECM  Block  I  would  use  the  ASPJ  and  the  AN/ALE-50  Towed 
Decoy.  These  are  existing  systems  that  the  Navy  would  use  until 
the  AN/ALQ-214  could  be  integrated. 

•  IDECM  Block  II  would  use  the  AN/ALQ-214  and  the 
AN/ALE-50  Towed  Decoy. 

•  IDECM  Block  III  would  use  the  AN/ALQ-214  and  AN/ALE-55 
Fiber  Optic  Towed  Decoy. 

Our  audit  focused  on  the  test  planning  and  requirements  for  the  ASPJ  and 
AN/ALQ-214  on-board  jammers. 

The  Navy  plans  to  have  an  inventory  of  366  AN/ALQ-214  systems  and 
10,980  AN/ALE-55  decoys.  The  Air  Force  plans  to  have  an  inventory  of 
92  AN/ALQ-214  systems,  3,069  AN/ALE-55  decoys  for  the  B-1B,  and  15,048 
AN/ALE-55  decoys  for  the  F-15.  The  prime  contractor  for  the  IDECM 
program  is  Sanders,  and  the  total  program  cost  is  estimated  to  be  $2.7  billion 
(then-year  dollars).  The  AN/ALQ-214  development  cost  is  $85.8  million  and 
the  procurement  cost  is  $1.1  billion  (458  units  at  $2.3  million  each). 

Objectives 


The  overall  objective  was  to  evaluate  the  test  planning  and  requirements  for  the 
ASPJ  and  the  AN/ALQ-214  on-board  jammers.  We  also  reviewed  the 
management  control  program  as  it  applied  to  the  audit  objective.  See 
Appendix  A  for  a  discussion  of  the  audit  scope  and  methodology  and  the  review 
of  the  management  control  program. 
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Operational  Requirements  of  the 
Integrated  Defensive  Electronic 
Countermeasures  Suite 

The  Navy  reduced  the  mission  reliability  requirements  from  the  level 
recommended  by  the  cost  and  operational  effectiveness  analysis.  The 
Navy  reduced  the  requirements  so  that  the  ASPJ,  which  the  Navy  plans 
to  use  as  the  on-board  jammer  portion  in  Block  I  of  the  IDECM  suite, 
could  pass  the  operational  evaluation  and  be  installed  on  the  F/A-18  E/F 
aircraft.  Furthermore,  the  AN/AFQ-214,  which  will  be  the  on-board 
jammer  for  Blocks  II  and  III  of  the  IDECM  suite,  will  be  tested  against 
the  same  mission  reliability  requirements.  By  reducing  the  mission 
reliability  rate,  the  Navy’s  logistical  support  requirement  may  have  to  be 
significantly  increased  in  order  to  accomplish  a  90  percent  operational 
availability  rate  for  the  system.  At  the  reduced  mission  reliability  rate, 
unscheduled  maintenance  may  be  required  as  much  as  2.5  times  more 
often  than  if  the  system  met  the  mission  reliability  rate  recommended  by 
the  cost  and  operational  effectiveness  analysis.  Additionally,  it  is 
unclear  whether  the  additional  protection  provided  by  the  on-board 
jamming  capability  justifies  the  investment  in  the  development, 
acquisition  and  logistical  support. 

Evolution  of  the  Integrated  Defensive  Electronic 
Countermeasures  System 


In  the  1992  operational  test  and  evaluation,  the  ASPJ  was  determined  not 
operationally  suitable  because  the  ASPJ  could  not  meet  the  requirement 
threshold  for  mission  reliability  and  built-in-test  effectiveness.  Mission 
reliability  is  a  measure  of  a  system’s  operational  effectiveness  and  is  stated  in 
terms  of  the  probability  of  completing  a  specific  mission  profile  or  as  a  function 
of  the  mean  time  between  operational  mission  failures.  Built-in-test 
effectiveness  is  the  probability  of  built-in-tests  correctly  detecting  a  fault.  See 
Appendix  B  for  more  detailed  information  on  the  results  of  the  ASPJ  operational 
test  and  evaluation  relating  to  mission  reliability  and  built- in-test  effectiveness. 

Because  the  system  did  not  meet  operational  suitability  requirements,  and  did 
not  meet  the  operational  effectiveness  requirement,  the  program  was  canceled 
and  production  was  terminated.  Termination  of  the  ASPJ  Program  left  the  Navy 
with  deficiencies  related  to  tactical  aircraft  self-protection.  On  September  8, 
1993,  the  Under  Secretary  of  Defense  for  Acquisition  (now  Acquisition, 
Technology,  and  Fogistics)  informed  the  chairmen  and  ranking  members  of  the 
Armed  Services  and  Appropriations  Committees  of  the  Senate  and  House  of 
Representatives  of  his  intent  to  direct  the  Navy  to  conduct  a  cost  and  operational 
effectiveness  analysis  of  the  F/A-18  electronic  warfare  suite. 
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Cost  and  Operational  Effectiveness  Analysis.  The  Navy  contracted  with  the 
Johns  Hopkins  University  Applied  Physics  Laboratory  to  perform  a  cost  and 
operational  effectiveness  analysis.  The  analysis  was  issued  on 
January  19,  1995.  The  analysis  addressed  the  first  four  deficiencies  in  the 
IDECM  Mission  Need  Statement  and  evaluated  four  electronic  warfare  suite 
alternatives  for  the  F/A-18  E/F.  The  analysis  concluded  that  the  electronic 
warfare  suites  that  included  a  Fiber  Optic  Towed  Decoy  were  the  most  effective 
suites.  This  analysis  also  showed  that  the  Fiber  Optic  Towed  Decoy  without  an 
on-board  jammer  was  generally  the  most  cost-effective  suite  and  that  it  became 
more  cost-effective  as  the  threat  environment  increased. 

Initiation  of  the  IDECM  Program.  The  IDECM  Program  began  as  an 
off-board  jammer,  specifically  the  Fiber  Optic  Towed  Decoy,  with  a 
complementary  on-board  radio  frequency  jammer  capability  as  a  possible 
upgrade.  The  Operational  Requirements  Document,  July  1,  1998,  established 
mission  reliability  for  the  system  at  94  percent,  as  recommended  in  the  cost  and 
operational  effectiveness  analysis.  However,  technical  issues  had  stressed  the 
program’s  schedule.  System  development  was  at  a  much  slower  pace  than 
expected,  leading  to  cost  and  schedule  overruns.  These  technical  issues 
included  towline  integrity  of  the  Fiber  Optic  Towed  Decoy,  reliability  of  the 
decoy,  and  Improved  Multi-Platform  Launch  Controller  used  to  launch  the 
decoys.  To  ensure  that  the  initial  deployments  of  the  F/A-18  E/F  aircraft  had 
self-protection,  the  Navy  decided  to  include  an  on-board  radio  frequency 
jammer  in  the  suite.  The  Director,  Air  Warfare  Division,  Office  of  the  Deputy 
Chief  of  Naval  Operations  (Resources,  Requirements  and  Assessments),  added 
the  requirement  for  on-board  jammers  to  provide  protection  in  the  event  that  the 
three  available  towed  decoys  were  expended  or  failed  to  operate  during  a 
mission. 

Cost  and  Operational  Effectiveness  Analysis  Revisited.  On  May  20,  1999, 
the  Johns  Hopkins  University  Applied  Physics  Laboratory  released  an  analysis 
entitled,  “Use  of  On-board  Transmitters  with  the  Integrated  Defensive 
Electronic  Countermeasures  System.  ”  The  document  reexamined  the 
Milestone  II  cost  and  operational  effectiveness  analysis  to  determine  whether  the 
previous  recommendations  should  be  revised.  The  analysis  determined  that, 
although  the  use  of  the  towed  decoy  was  the  most  effective  alternative  against 
threats,  consideration  should  be  given  to  using  on-board  transmitters  in 
conjunction  with  the  towed  decoy  system  as  part  of  the  F/A-18  electronic 
warfare  suite.  The  analysis  also  noted  that  some  assets  were  available  from  the 
terminated  ASPJ  production.  On-board  transmitters  provide  an  additional 
capability  to  counter  radio  frequency  guided  missile  threat  when: 

•  the  radio  frequency  of  the  threat  is  below  that  of  the  Fiber  Optic 
Towed  Decoy,  but  within  that  of  the  on-board  system; 

•  the  aircraft  does  not  have  a  decoy  deployed; 

•  a  decoy  fails  and  a  new  decoy  has  not  yet  become  operational; 
and 

•  all  of  the  decoys  have  been  expended. 
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The  cost  and  operational  effectiveness  analysis  viewed  the  on-board  jamming 
capability  as  a  redundant  system,  intended  to  supplement  the  Fiber  Optic  Towed 
Decoy  and  not  to  be  the  primary  system  for  providing  electronic  warfare 
protection.  The  cost  and  operational  effectiveness  analysis  stated: 

Assuming  that  the  aircraft  survives  but  loses  a  decoy  each  time  it  is 
engaged,  it  will  have  lost  all  of  its  decoys  by  the  end  of  the  third 
engagement.  If  the  aircraft  has  onboard  transmitters,  it  will  have 
some  capability  to  counter  subsequent  threats  with  radio  frequency 
countermeasures  (RFCM).  However,  discussions  with  operational 
personnel  during  the  COEA  [Cost  and  Operational  Effectiveness 
Analysis]  indicated  that  they  would  most  likely  exit  the  threat  area  as 
quickly  as  possible  after  the  first  time  they  were  engaged.  This  was 
partially  driven  by  the  fact  that  their  tactic  for  defeating  missiles  in  the 
endgame  included  a  maximum-g  maneuver.  Because  the  aircraft  is 
more  maneuverable  without  ordnance,  they  would  generally  jettison 
their  air-to-ground  ordnance  prior  to  performing  the  endgame 
maneuver.  If  the  aircraft  survives,  there  would  not  be  any  reason  for 
the  aircraft  to  continue  its  mission  in  the  threat  area  other  than  to 
provide  support  for  aircraft  that  had  not  jettisoned  their  ordnance. 

Thus,  the  probability  that  the  aircraft  would  be  engaged  more  than 
three  times  during  a  mission  was  minimal. 

Addition  of  On-Board  Jammer  Capability.  The  Navy  had  about  40  ASPJs 
that  could  be  used  to  provide  electronic  warfare  protection  for  the  F/A-18  E/F. 
The  Navy  considers  the  use  of  the  ASPJ  as  an  interim  solution  until  Block  II  is 
implemented.  The  Navy  established  a  phased  approach  to  provide  the  electronic 
warfare  protection  for  the  F/A-18  E/F.  Three  sequential  phases  for  deployment 
of  on-board  jammers  and  towed  decoys  were  to  be  used. 

•  IDECM  Block  I  would  use  the  existing  inventory  of  ASPJ  and  the 
existing  AN /ALE-50  Towed  Decoy  until  the  AN/ALQ-214  could 
be  integrated. 

•  IDECM  Block  II  would  use  the  AN/ALQ-214  and  the 
AN/ALE-50  Towed  Decoy. 

•  IDECM  Block  III  would  use  the  AN/ALQ-214  and  AN/ALE-55 
Fiber  Optic  Towed  Decoy. 

Block  III  supposedly  will  provide  increased  effectiveness  through  the  use  of 
integrated  Fiber  Optic  Towed  Decoy  techniques  and  increased  reliability  and 
would  be  eventually  replace  the  Block  I  and  II  configurations  on  the  F/A-18  E/F 
aircraft. 
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Requirements  and  Test  Planning  for  the  Integrated  Defensive 
Electronic  Countermeasures  System 


Development  of  the  Operational  Requirements  Documents  for  the  On-board 
Jammers  for  IDECM.  Operational  testing  was  required  before  the  Navy  could 
obtain  approval  to  install  the  ASPJ  on  the  F/A-18  E/F  aircraft  as  Block  I  of  the 
IDECM  Program.  For  the  system  to  be  tested,  the  Navy  needed  a  test  plan, 
which  in  turn  required  a  Test  and  Evaluation  Master  Plan.  Since  the  Test  and 
Evaluation  Master  Plan  establishes  tests  to  determine  whether  the  system 
satisfies  criteria  established  in  the  Operational  Requirements  Document,  the 
Navy  wrote  a  new  Operational  Requirements  Document  that  incorporated 
equipment  that  it  owned  and  that  had  previously  failed  operational  test  and 
evaluation  requirements  in  1992. 

Test  Plan  for  the  Operational  Evaluation  of  the  On-Board  Jammers.  Test 
Planning  for  the  ASPJ  and  the  AN/ALQ-214  on-board  jammers  to  be  used  in  the 
IDECM  suite  was  extensive.  A  Test  and  Evaluation  Master  Plan  was  developed 
for  each  block  of  the  IDECM  suite.  The  IDECM  Test  and  Evaluation  Master 
Plan  states  that  successful  demonstration  of  the  operational  effectiveness  and 
operational  suitability  requirements  established  in  the  On-Board  Jammer 
Operational  Requirements  Document  is  required  prior  to  fleet  installation  and 
deployment. 

Operational  Effectiveness.  In  May  1992,  the  Director  of  Operational 
Test  and  Evaluation  assessed  ASPJ  as  not  being  operationally  effective  because 
it  did  not  meet  the  requirement  threshold  value  for  increasing  the  survivability 
of  an  F/A-18  strike  force  over  that  of  a  non- ASPJ  baseline  F-18  strike  force. 
Although  the  requirements  are  not  as  stringent  as  those  in  the  1992  operational 
test  and  evaluation  of  ASPJ,  the  IDECM  test  plans  require  that  the  IDECM 
on-board  jammers  demonstrate  that  they  are  measurably  more  operationally 
effective  than  the  AN/ALQ-126B  on-board  jammer  and  an  aircraft  without  a 
self-protection  capability. 

Operational  Suitability.  The  1992  operational  test  and  evaluation 
determined  that  the  ASPJ  was  not  operationally  suitable  because  it  could  not 
meet  the  requirement  thresholds  for  mission  reliability  and  built-in-test 
effectiveness.  The  following  table  summarizes  the  reduction  in  requirements 
from  the  1992  ASPJ  Test  and  Evaluation  Master  Plan  and  the  current 
Operational  Requirements  Document  for  IDECM.  If  the  ASPJ  had  been  tested 
against  the  reduced  requirements,  rather  than  the  requirements  it  was  tested 
against  in  1992,  the  ASPJ  would  have  been  found  to  be  operationally  suitable 
and  the  program  might  not  have  been  terminated. 
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Differences  Between  the  1992  ASPJ  and  IDECM  On-Board  Jammers 
Operational  Suitability  Test  Requirements 

Requirement 

ASPJ 

IDECM  Onboard 

Jammers 

Mission  Reliability  (percent) 

95 

86 

Mean  Corrective  Maintenance 
Time  for  Operational 

Mission  Failures  (minutes) 

45 

120 

Probability  of  Correct 

Detection  (percent) 

90 

80 

Built-in-Test  False 

Alarm  Rate 

10  percent* 

5  per  100  flight  hours 

*In  the  1992  Operational  Test  and  Evaluation,  the  10  percent  requirement  for  Built-in-Test 

False  Alarm  Rate  equated  to  2  incorrect  removals  per  100  flight  hours. 

A  mission  reliability  threshold  reflects  the  recommended  operational  level  at 
which  the  aircraft’s  electronic  warfare  suite  should  perform  in  the  threat 
environment.  The  Navy  reduced  the  mission  reliability  requirement  from 
94  percent,  as  recommended  in  the  cost  and  operational  effectiveness  analysis, 
to  86  percent  in  the  Operational  Requirements  Document.  The  mission 
reliability  requirement  was  reduced  due  to  the  inclusion  of  the  improved  multi¬ 
platform  launch  controller,  launcher,  on-board  transmitters,  and  other  aircraft 
hardware  in  scoring  test  failures. 

The  IDECM  Program  Management  Office  calculated  that  the  reliability  of  the 
original  configuration  of  the  system  could  achieve  the  recommended  94  percent 
mission  reliability  threshold.  This  configuration  comprised  only  the  technique 
generator,  fiber  optic  towed  decoy,  and  a  signal  conditioning  assembly. 
However,  the  independent  testers  of  the  system  indicated  that  testing  the 
reliability  of  only  the  off-board  portion  of  the  system  would  not  accurately 
represent  the  mission  reliability  of  the  aircraft’s  electronic  warfare  suite. 
Therefore,  additional  components  such  as  the  on-board  transmitter,  the 
Improved  Multi-Platform  Launch  Controller,  and  other  components  were 
included  in  the  predicted  reliability  calculations.  After  incorporating  those  items 
into  the  reliability  model,  the  predicted  mission  reliability  of  the  aircraft’s 
electronic  warfare  dropped  below  the  94  percent  threshold.  Discussions 
between  the  IDECM  Program  Management  Office  and  the  independent  testers 
indicated  that  Block  I  of  the  IDECM  program,  as  configured,  would  not  pass  if 
the  system  was  tested  against  the  mission  reliability  requirement  of  94  percent. 
Subsequently,  the  Navy  revised  the  mission  reliability  requirement  in  the  revised 
Operational  Requirement  Document  to  reduce  the  mission  reliability  to 
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86  percent,  which  is  a  mean  time  between  the  operational  mission  failure 
requirement  of  12  hours  for  the  new  configuration  of  the  IDECM  system.  As 
such,  the  requirement  in  the  Operational  Requirements  Document  and  test 
planning  documentation  represents  what  the  entire  system  configuration  must 
achieve  under  the  independent  testers  scoring  philosophy.  Officials  from  the 
IDECM  Program  Management  Office  stated  that  the  86  percent  requirement  for 
mission  reliability  is  not  the  performance  level  at  which  the  system  should 
perform  if  only  essential  countermeasure  components,  the  AN/ALQ-214  and  the 
AN/ALE-55  Fiber  Optic  Towed  Decoy,  were  considered  in  scoring  mission 
reliability. 

The  Navy  also  eased  the  requirement  for  built-in-test  effectiveness  for  incorrect 
component  removals  caused  by  built-in-test  fault  indications  in  Blocks  I,  II,  and 
III  of  IDECM  to  five  removals  for  each  100  hours  of  flight  time.  In  the 
1992  operational  test  and  evaluation,  the  ASPJ  failed  with  two  incorrect 
removals  per  100  flight  hours. 

In  addition  to  mission  reliability  and  built-in-test  effectiveness,  the  Navy 
changed  the  operational  suitability  requirement  for  mean  corrective  maintenance 
time  for  operational  mission  failures  from  45  minutes  to  2  hours.  The  IDECM 
Program  Management  Office  stated  that  the  requirement  was  changed  due  to  the 
addition  of  hardware  and  in  response  to  lessons  learned  from  previous 
operational  tests  where  support  equipment  used  to  troubleshoot  failures 
increased  the  corrective  maintenance  time. 

Test  Requirements  for  the  AN/ALQ-214  On-Board  Jammer.  The  on-board 
jammer  for  the  Block  II  and  Block  III  versions  of  the  IDECM  suite,  the 
AN/ALQ-214,  will  be  tested  against  the  same  requirements  used  to  evaluate  the 
ASPJ  in  Block  I  testing.  The  AN/ALQ-214  design  is  based  on  the  ASPJ  design 
and  has  about  60  percent  commonality  with  the  ASPJ. 

Effect  of  Reduced  IDECM  Mission  Reliability.  We  were  unable  to  determine 
any  impact  that  the  reduction  in  mission  reliability  would  have  on  loss  of 
aircraft.  However,  the  reduction  in  mission  reliability  will  have  a  significant 
impact  on  operations  and  maintenance  costs. 

Aircraft  Attrition.  The  cost  and  operational  effectiveness  analysis 
released  in  January  1995  by  the  Johns  Hopkins  University  Applied  Physics 
Laboratory  also  examined  the  impact  of  reliability  on  the  expected  number  of 
aircraft  losses.  The  analysis  showed  that  maintaining  mean  time  between 
operational  mission  failures  of  30  hours  or  higher  produced  no  changes  in 
predicted  aircraft  attrition  rates.  For  a  1.8-hour  mission,  a  mean  time  between 
operational  mission  failure  of  30  hours  equates  to  a  mission  reliability  of 
94  percent.  The  cost  and  operational  effectiveness  analysis  stated  that  a 
significant  increase  in  attrition  of  aircraft  would  occur  if  the  mission  reliability 
were  less  than  94  percent.  However,  discussions  with  officials  at  the  Johns 
Hopkins  University  Applied  Physics  Laboratory  indicated  that  a  lower  mission 
reliability  would  have  a  negligible  effect  on  the  attrition  of  aircraft.  Using  the 
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same  formula,  we  analyzed  the  impact  of  an  86  percent  mission  reliability  on  the 
expected  number  of  aircraft  losses.  The  analysis  showed  that  reducing  the  mean 
time  between  operational  mission  failures  produced  only  slight  changes  in  the 
predicted  number  of  aircraft  losses. 

Impact  on  Logistical  Support.  The  reduction  in  mission  reliability  will 
impact  the  logistical  support  for  the  IDECM.  At  an  86  percent  mission 
reliability  rate,  or  a  mean  time  between  operational  mission  failure  of  12  hours, 
unscheduled  maintenance  may  be  required  as  much  as  2.5  times  more  often  than 
if  the  system  met  the  recommended  mission  reliability  rate.  The  Operational 
Requirements  Document  states  that  the  objective  value  for  mean  time  between 
operational  mission  failure  is  150  hours.  That  value  would  be  more  in  line  with 
the  mean  time  between  operational  mission  failure  for  other  countermeasure 
systems;  however,  we  have  not  seen  any  progression  towards  measuring  a 
higher  mean  time  between  operational  mission  failure  for  IDECM  in  the 
requirements  or  test  planning  documentation.  An  official  at  the  IDECM 
Program  Management  Office  detachment  in  Jacksonville,  Florida,  informed  us 
that  the  AN/ALQ-126B  on-board  jammer,  which  the  IDECM  on-board  jammers 
will  be  tested  against,  was  achieving  a  mean  time  between  failure  rate  of 
223  hours.  However,  the  official  stated  that  the  operational  availability  of  the 
AN/ALQ-126B  was  78  percent  because  they  are  only  used  if  the  fleet  is  going  to 
combat.  The  AN/ALE-50  towed  decoy  requirement  for  mean  time  between 
operational  mission  failure  that  will  be  measured  in  the  Block  I  Operational  Test 
and  Evaluation  is  100  hours. 

Although  the  Navy  reduced  the  mission  reliability  of  the  IDECM,  it  kept  the 
requirement  for  operational  availability  of  the  IDECM  suite  at  90  percent. 
Operational  availability  is  the  degree  that  equipment  or  weapons  systems  are 
expected  to  work  properly.  Operational  availability  is  one  of  the  key 
performance  parameters  in  the  Operational  Requirements  Document.  Key 
performance  parameters  are  those  so  significant  that  failure  to  meet  the 
threshold  value  of  performance  can  be  a  cause  for  the  concept  or  system  to  be 
reevaluated  or  the  program  to  be  reassessed  or  terminated.  Although  the  Navy 
might  be  able  to  accomplish  a  90  percent  operational  availability  in  a  short  test 
period,  we  do  not  believe  that  the  Navy  will  be  willing  to  fund  the  additional 
maintenance  actions  and  obtain  the  spare  parts  needed  to  keep  the  IDECM 
90  percent  operationally  available  if  the  system  only  has  a  mean  time  between 
operation  mission  failure  of  12  hours.  Accordingly,  the  Navy’s  logistical 
support  requirement  may  have  to  be  significantly  increased  to  accomplish  a 
90  percent  operational  availability  rate  for  the  system.  If  the  Navy  does  not 
provide  the  logistics  support  needed  to  accomplish  a  90  percent  operational 
availability  rate,  operational  effectiveness  of  the  IDECM  system  will  be  further 
reduced. 

Test  Results.  We  recognize  that  the  mission  reliability  requirement  defines  the 
minimum  acceptable  performance  standard.  The  mission  reliability  requirement 
neither  predicts  the  actual  system  or  component  failure  rates,  nor  determines  the 
anticipated  logistic  support  requirements.  Failure  rates  are  predicted  based  on 
analysis  and  testing  of  analogous  systems  and  components,  testing  of  actual 
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system  hardware,  and  engineering  estimates  when  appropriate.  Those  predicted 
failure  rates  are  then  used  to  determine  the  expected  level  of  logistic  support 
required. 


Conclusion 


Although  the  ASPJ  Program  was  terminated  8  years  ago,  the  Navy  intends  to 
use  the  ASPJ  in  Block  I  of  the  IDECM  to  satisfy  the  F/A-18  E/F  requirement 
for  self-protection  capability  against  radio  frequency  and  infrared  surface-to-air 
and  air-to-air  threats.  Even  though  the  Navy  reduced  the  mission  reliability 
requirements  established  by  the  cost  and  operational  effectiveness  analysis, 
Block  I  of  IDECM  should  provide  the  F/A-18  E/F  with  an  amount  of  protection 
that  is  not  otherwise  available  against  the  threat.  Since  Block  I  will  use  the 
existing  inventory  of  ASPJ  and  AN/ALE-50  Towed  Decoy,  there  will  be  no 
additional  system  procurement  costs,  only  support  costs. 

However,  the  on-board  jamming  capability— the  ASPJ  and  the  AN/ALQ-2 14- 
are  redundant  systems  intended  to  supplement  the  Fiber  Optic  Towed  Decoy, 
not  to  be  primary  systems  for  providing  electronic  warfare  protection.  Given 
the  reduced  mission  reliability  rate,  it  is  unclear  how  much  more  protection  is 
being  provided  by  these  systems  and  whether  the  Navy  will  fund  the  additional 
logistical  support  costs. 

Management  Comments  on  the  Finding  and  Audit  Response 


Navy  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy  for  Air 
Programs  (the  Deputy  Assistant  Secretary)  nonconcurred  with  the  finding, 
stating  that  the  mission  reliability  requirement  was  not  reduced  to  allow  the 
Airborne  Self-Protection  Jammer  to  pass  the  Operational  Evaluation  for  Block  I. 
The  mission  reliability  requirement  contained  in  the  Operational  Requirements 
Document  for  the  Integrated  Defensive  Electronic  Countermeasures  was  based 
upon  systems  engineering  analysis. 

The  Deputy  Assistant  Secretary  did  not  agree  that  the  Navy’s  logistical  support 
requirement  might  have  to  be  significantly  increased  because  of  the  reduction  in 
the  mission  reliability  requirement.  The  Deputy  Assistant  Secretary  stated  that 
current  logistics  cost  requirements  were  based  upon  the  system  (including  on¬ 
board  jammers)  as  it  was  intended  to  be  fielded. 

The  Deputy  Assistant  Secretary  also  did  not  agree  that  it  was  unclear  whether 
the  additional  protection  provided  by  on-board  jamming  capability  justifies  the 
investment  in  the  development,  acquisition,  and  logistical  support.  The  Deputy 
Assistant  Secretary  provided  circumstances  under  which  on-board  jammers  are 
expected  to  improve  survivability  of  the  aircraft  and  stated  that,  as  non- 
developmental  items,  the  on-board  jammers  have  minimal  impact  in  terms  of 
development,  acquisition,  and  logistics  costs. 
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For  the  full  text  of  the  Navy  comments,  see  the  Management  Comments  section 
of  this  report. 


Audit  Response.  On  January  19,  1995,  the  Johns  Hopkins  University  Applied 
Physics  Laboratory  issued  a  cost  and  operational  effectiveness  analysis  on 
alternative  electronic  warfare  suites.  The  analysis  recommended  that  the 
preferred  alternative,  regardless  of  whether  that  system  utilized  an  on-board 
jammer,  have  a  mission  reliability  requirement  of  94  percent.  The  system  being 
developed  by  the  Navy  does  not  meet  that  requirement.  Furthermore, 
discussions  with  officials  of  the  Office  of  the  Deputy  Chief  of  Naval  Operations 
(Resources,  Requirements,  and  Assessments)  indicated  that  the  Navy  was  aware 
that  Block  I  of  IDECM  (the  ASPJ  and  the  existing  AN/ALE-50  Towed  Decoy) 
would  be  destined  for  failure  if  the  system  was  required  to  meet  a  94  percent 
mission  reliability. 

The  Deputy  Assistant  Secretary  did  not  address  the  increased  logistical  support 
requirement  as  a  result  of  reducing  the  requirement,  but  rather  the  current 
logistics  cost  requirements  now  that  the  Navy  has  reduced  the  mission  reliability 
requirement. 

We  agree  with  the  Deputy  Assistant  Secretary,  as  stated  in  the  report,  that  on¬ 
board  transmitters  would  provide  an  additional  capability  to  counter  radio 
frequency  guided  missile  threat  in  certain  circumstances.  We  also  agree  that 
using  the  ASPJ  as  the  on-board  jammer  for  Block  I  of  IDECM  will  have 
minimal  impact  in  terms  of  development  and  acquisition  costs.  However, 
development  and  procurement  of  the  AN/ALQ-214  for  Blocks  II  and  III  of 
IDECM  is  significant.  The  AN/ALQ-214  development  cost  is  $85.8  million 
and  the  procurement  cost  is  $1.1  billion  (458  units  at  $2.3  million  each),  which 
represents  about  44  percent  of  the  $2.7  billion  total  program  costs  for  IDECM. 

Recommendations,  Management  Comments,  and  Audit 
Response 


We  recommend  that  the  Program  Manager,  Integrated  Defensive  Electronic 
Countermeasures,  prior  to  proceeding  with  Block  II  of  the  Program: 

1.  Determine  the  logistics  support  cost  to  maintain  the  on-board 
jamming  capability  based  on  the  results  of  the  Operational  Test  and 
Evaluation. 

Navy  Comments.  The  Deputy  Assistant  Secretary  concurred,  stating  that  the 
Navy  intends  to  review  the  logistics  support  costs  in  the  normal  course  of 
preparing  for  the  Milestone  III  decision  planned  for  the  third  quarter  of 
FY  2002.  For  the  full  text  of  the  Navy  comments,  see  the  Management 
Comments  section  of  this  report. 


11 


2.  Reassess  whether  the  on-board  jamming  capability  provides  an 
amount  of  added  protection  that  justifies  its  development,  acquisition  and 
logistical  support  costs. 

Navy  Comments.  The  Deputy  Assistant  Secretary  nonconcurred,  stating  that 
the  Johns  Hopkins  University  Applied  Physics  Laboratory  assessed  the  cost 
versus  benefit  of  having  an  on-board  jamming  capability  in  May  1999  and  that 
another  assessment  was  not  necessary.  For  the  full  text  of  the  Navy  comments, 
see  the  Management  Comments  section  of  this  report. 

Audit  Response.  The  Johns  Hopkins  University  Applied  Physics  Laboratory 
performed  two  cost  and  operational  effectiveness  analyses  for  IDECM.  The 
January  1995  analysis  concluded  that  the  electronic  warfare  suites  that  included 
a  Fiber  Optic  Towed  Decoy  were  the  most  effective  suites.  That  analysis  also 
showed  that  the  Fiber  Optic  Towed  Decoy  without  an  on-board  jammer  was 
generally  the  most  cost-effective  suite  and  that  it  became  more  cost-effective  as 
the  threat  environment  increased.  The  May  1999  analysis  summarized  the 
results  of  the  January  1995  analysis  as  follows: 

Because  the  advantages  of  onboard  transmitters  were  assessed  to  be 
somewhat  limited  and  there  was  a  significant  cost  increase  associated 
with  the  addition  of  onboard  transmitters,  the  COE  A  [cost  and 
operational  effectiveness  analysis]  recommended  that  the  onboard 
transmitters  should  not  be  included  in  the  basic  EW  [electronic 
warfare]  suite. 

The  May  1999  analysis  determined  that,  although  the  use  of  the  towed  decoy 
was  the  most  effective  alternative  against  threats,  consideration  should  be  given 
to  using  on-board  transmitters  in  conjunction  with  the  towed  decoy  system  as 
part  of  the  F/A-18  electronic  warfare  suite.  However,  the  May  1999  analysis 
did  not  address  the  cost  versus  the  additional  benefit  that  on-board  transmitters 
would  provide. 
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Appendix  A.  Audit  Process 

Scope  and  Methodology 


We  reviewed  documentation  dated  from  May  1992  through  September  2000. 

We  used  criteria  in  the  DoD  Regulation  5000. 2-R  to  perform  the  audit.  To 
accomplish  the  audit  objectives,  we  took  the  following  steps: 

•  determined  that  a  valid  requirement  still  existed  for  a  replacement 
to  the  AN/ALQ-126B  jammer; 

•  determined  the  users  had  approved  the  operational  requirements; 

•  determined  the  IDECM  program’s  operational  test  and  evaluation 
included  all  of  the  system’s  operational  requirements,  as  defined 
in  the  Operational  Requirements  Document;  and 

•  reviewed  management  controls  related  to  the  audit  objective. 

Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Use  of  Technical  Assistance.  We  used  mechanical  and  electronics  engineers 
from  the  Technical  Assessment  Division,  Office  of  the  Inspector  General,  DoD, 
to  assist  us  in  analyzing  and  evaluating  the  operational  suitability  testing 
requirements  for  IDECM. 

Audit  Period  and  Standards.  We  performed  this  program  audit  from  July  2000 
through  November  2000,  in  accordance  with  standards  issued  by  the 
Comptroller  General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD,  and  included  such  tests  of  management  controls  as  deemed 
necessary. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  We  also  visited  or  contacted  individuals  and 
organizations  at  the  Johns  Hopkins  University  Applied  Physics  Laboratory. 
Further  details  are  available  upon  request. 

DoD-Wide  Corporate  Level  Government  Performance  and  Results  Act 
Coverage.  In  response  to  the  Government  Performance  and  Results  Act,  the 
Secretary  of  Defense  annually  establishes  DoD-wide  corporate  level  goals, 
subordinate  performance  goals,  and  performance  measures.  This  report  pertains 
to  achievement  of  the  following  goal,  subordinate  performance  goal,  and 
performance  measure. 

•  FY  2001  DoD  Corporate-Level  Goal  2:  Prepare  now  for  an 
uncertain  future  by  pursuing  a  focused  modernization  effort  that 
maintains  U.S.  qualitative  superiority  in  key  warfighting  capabilities. 
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Transform  the  force  by  exploiting  the  Revolution  in  Military  Affairs, 
and  reengineer  the  Department  to  achieve  a  21st  century 
infrastructure .  (01-DoD-02) 

•  FY  2001  Subordinate  Performance  Goal  2.4:  Meet  combat  forces’ 
needs  smarter  and  faster,  with  products  and  services  that  work  better 
and  cost  less,  by  improving  the  efficiency  of  DoD  acquisition 
processes.  (Ol-DoD-2.4) 

•  FY  2001  Performance  Measure  2.4.3:  Successful  Completion  of 
System  Operational  Test  and  Evaluation  (OT&E)  Events. 

(Ol-DoD-2.4.3) 

General  Accounting  Office  High-Risk  Area.  The  General  Accounting 
Office  has  identified  several  high-risk  areas  in  the  DoD.  This  report 
provides  coverage  of  the  Defense  Weapons  Systems  Acquisition 
high-risk  area. 

Management  Control  Program  Review 


DoD  Directive  5010.38,  “Management  Control  (MC)  Program,”  August  26, 
1996,  and  DoD  Instruction  5010.40,  “Management  Control  (MC)  Program 
Procedures,”  August  28,  1996,  require  DoD  managers  to  implement  a 
comprehensive  system  of  management  controls  that  provides  reasonable 
assurance  that  programs  are  operating  as  intended  and  to  evaluate  the  adequacy 
of  the  controls. 

Scope  of  the  Review  of  the  Management  Control  Program.  In  accordance 
with  DoD  Directive  5000.1,  “Defense  Acquisition,”  March  15,  1996,  and  DoD 
Regulation  5000. 2-R,  Mandatory  Procedures  for  Major  Defense  Acquisition 
Programs  (MDAPs)  and  Major  Automated  Information  System  Acquisition 
Programs  (MAIS),”  May  11,  1999,  acquisition  managers  are  to  use  program 
cost,  schedule,  and  performance  parameters  as  control  objectives  to  implement 
the  requirements  of  DoD  5010.38.  Accordingly,  we  limited  our  review  to 
management  controls  directly  related  to  test  planning  of  the  IDECM. 

In  evaluating  the  management  control  process,  we  reviewed  the  risk- 
management  program  to  determine  the  level  of  risk  that  the  officials  assigned  to 
aspects  of  IDECM.  We  also  reviewed  the  FY  1999  and  2000  Annual 
Statements  of  Assurance  for  the  Navy  to  determine  whether  any  weaknesses  had 
been  reported  relating  to  the  IDECM  program.  Because  we  did  not  identify  a 
material  management  control  weakness,  we  did  not  assess  management’s 
self-evaluation. 

Adequacy  of  Management  Controls.  Management  controls  were  adequate  in 
that  we  did  not  identify  any  systemic  management  control  weakness  applicable 
to  our  audit  objective. 
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Prior  Coverage 

During  the  last  5  years,  no  reports  have  been  issued  related  to  the  test  planning 
and  requirements  for  the  on-board  jammers  for  the  F/A-18  E/F  aircraft. 
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Appendix  B.  Mission  Reliability  and  Built-in- 

Test  Effectiveness  of  Airborne 
Self-Protection  Jammer 


Mission  Reliability 


Mission  reliability  is  a  measure  of  a  system’s  operational  effectiveness  and  is 
stated  in  terms  of  the  probability  of  completing  a  specific  mission  profile  or  as  a 
function  of  the  mean  time  between  operational  mission  failures.  For  a  given 
mission  of  time,  mission  reliability  is  calculated  by  taking  the  natural  logarithm 
of  the  negative  quotient  of  the  mission  of  time  divided  by  the  mean  time 
between  operational  mission  failure.  The  mean  time  between  operational 
mission  failure  is  calculated  by  dividing  the  total  system  operating  time  by  the 
number  of  operational  mission  failures.  System  operating  time  includes  only  the 
time  the  system  is  operating  and  being  stressed  under  operational  loads.  It  does 
not  include  standby  time.  For  aircraft,  system  operating  time  is  measured  from 
the  attempt  to  start  the  aircraft  with  the  intent  to  perform  a  mission  until 
shutdown.  An  operational  mission  failure  is  one  that  prevents  the  system  from 
performing  one  or  more  mission  essential  functions.  In  the  1992  Operational 
Test  and  Evaluation  of  the  ASPJ,  the  requirement  for  mean  time  between 
operational  mission  failure  was  33.3  hours,  which  equates  to  a  95  percent 
mission  reliability.  In  the  test,  the  ASPJ  was  able  to  achieve  22.7  hours,  a 
92  percent  mission  reliability. 

Built-in-Test  Effectiveness 


Built-in-test  effectiveness  is  the  probability  of  built-in-tests  correctly  detecting  a 
fault.  Built-in-test  inadequacies  were  a  significant  factor  in  the  ASPJ  failing  to 
meet  mission  reliability  requirements  in  the  1992  Operational  Test  and 
Evaluation.  The  probability  of  built-in-tests  correctly  detecting  a  fault  was 
64  percent  versus  a  criterion  of  at  least  90  percent.  The  removal  of  components 
where  built-in-tests  incorrectly  indicated  that  the  component  was  faulty  far 
exceeded  the  criteria  of  10  percent.  More  than  48  percent  of  the  components 
removed  as  a  result  of  built- in-test  fault  indications  were  not  actual  failures. 

The  ASPJ  failed  the  operational  test  and  evaluation  with  2  incorrect  removals 
per  100  flight  hours.  It  was  stated  in  the  “Operational  Test  and  Evaluation 
Report  on  the  AN/ALQ-165  (V)  Airborne  Self-Protection  Jammer  (ASPJ)” 
December  4,  1992,  that: 

...at  the  organizational  level,  pilots  and  maintenance  personnel  cannot 
differentiate  between  an  actual  hardware  failure  or  a  software  fault, 
including  BIT-  [built-in-test]  related  problems.  The  critical  failure 
criterion  could  have  been  met  if  the  false  removals  had  not  occurred. 
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Appendix  C.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Acquisition,  Technology,  and  Logistics) 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 
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Director,  Operational  Test  and  Evaluation 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Manpower  and  Reserve  Affairs) 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 

Chief  of  Naval  Operations 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 

Program  Executive  Officer,  Tactical  Aircraft  Programs 

Program  Manager,  Integrated  Defensive  Electronic  Countermeasures 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Contract  Management  Agency 
Director,  Defense  Logistic  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
Defense  Systems  Management  College 
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Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Efficiency,  Financial  Management,  and 
Intergovernmental  Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology  and  Procurement  Policy,  Committee  on 
Government  Reform 
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Department  of  the  Navy  Comments 


DEPARTMENT  OF  THE  NAVY 

OFFICE  OF  THE  ASSISTANT  SECRETARY 
RESEARCH,  DEVELOPMENT  AND  ACQUISITION 
1000  NAVY  PENTAGON 
WASHINGTON  DC  20350-1000 

1  6  FES  2001 


MEMORANDUM  FOR  THE  DEPARTMENT  OF  DEFENSE  ASSISTANT 
INSPECTOR  GENERAL  FOR  AUDITING 

Subj :  AUDIT  REPORT  ON  THE  ON-BOARD  JAMMERS  FOR  THE  INTEGRATED 
DEFENSIVE  ELECTRONIC  COUNTERMEASURES 
(PROJECT  NO.  D2000AL-0243) 

Ref:  (a)  DODIG  memo  of  7  Dec  2000 

Enel:  (1)  Department  of  the  Navy  Response 

In  response  to  reference  (a) ,  our  comments  are  provided  in 
enclosure  (1) .  We  concur  with  only  one  recommendation,  but  each 
finding  and  recommendation  is  addressed  in  the  enclosure. 


Deputy  Assistant  Secretary  of  the 
Navy  for  Air  Programs 


Copy  to: 

NAVINSGEN  (42) 
COMNAVAIRSYSCOM  (AIR-09G) 


DEPARTMENT  OF  THE  NAVY  RESPONSE  TO 
DODIG  DRAFT  AUDIT  REPORT  OF  07  DECEMBER  2000 
“ON-BOARD  JAMMERS  FOR  THE  INTEGRATED  DEFENSIVE 
ELECTRONIC  COUNTERMEASURES” 

(PROJECT  NO.  D2000AL-0243) 

FINDING: 

The  Navy  reduced  the  mission  reliability  requirements  from  the  level  recommended  by 
the  cost  and  operational  effectiveness  analysis.  The  Navy  reduced  the  requirements  so 
that  the  ASPJ,  which  the  Navy  plans  to  use  as  the  on-board  jammer  portion  in  Block  I  of 
the  IDECM  suite,  could  pass  the  operational  evaluation  and  he  installed  on  the  F/A-18 
E/F  aircraft.  Furthermore,  the  AN/ALQ-214,  which  will  be  on  the  on-board  jammer  for 
Blocks  II  and  III  of  the  IDECM  suite,  will  be  tested  against  the  same  mission  reliability 
requirements.  By  reducing  the  mission  reliability  rate,  the  Navy’s  logistical  support 
requirement  may  have  to  be  significantly  increased  in  order  to  accomplish  a  90  percent 
Operational  Availability  rate  for  the  system.  At  the  reduced  mission  reliability  rate, 
unscheduled  maintenance  may  be  required  as  much  as  2.5  times  more  often  than  if  the 
system  met  the  mission  reliability  rate  recommended  by  the  cost  and  operational 
effectiveness  analysis.  Additionally,  it  is  unclear  whether  the  additional  protection 
provided  by  the  on-board  jamming  capability  justifies  the  investment  in  the  development, 
acquisition  and  logistical  support. 

Department  of  the  Navy  Response: 

We  do  not  concur  with  the  finding  that  the  Navy  reduced  the  Integrated  Defensive 
Electronic  Countermeasures  (IDECM)  mission  reliability  requirement  to  allow  Airborne 
Self-protection  Jammer  (ASPJ)  to  pass  the  IDECM  Block  1  Operational  Evaluation.  The 
IDECM  Operational  Requirements  Document  (ORD)  mission  reliability  requirement  was 
based  upon  a  systems  engineering  analysis  that  is  traceable  to  the  total  F/A-18E/F 
mission  reliability  requirement.  Each  system,  including  the  AN/ALQ-214  (consisting  of 
only  the  techniques  generator,  signal  conditioning  assembly,  and  towed  decoy  at  the 
time)  was  allocated  a  reliability  requirement  that  contributed  to  the  total  platform 
requirement. 

While  the  IDECM  program  was  being  restructured  in  1999,  the  Navy  opted  to  include 
on-board  transmitters  in  the  AN/ALQ-214  system  configuration.  The  selected 
acquisition  strategy  was  to  incorporate  existing  high  and  low  band  transmitters  (ASPJ 
WRA  4  and  WRA  5)  into  the  AN/ALQ-214  as  non-developmental  items.  A  separate  On- 
Board  Jammer  ORD  was  written  to  establish  the  requirements  for  the  ALQ-214  system 
with  the  added  transmitters.  The  mission  reliability  requirement  was  determined  by 
applying  the  appropriate  engineering  analysis  to  the  anticipated  threat  environment,  the 
system  configuration  being  developed,  and  the  current  operational  test  scoring 
philosophy.  The  final  number  does  not  pertain  to  just  the  on-board  transmitters,  but  to 
the  electronic  warfare  (EW)  suite  as  a  whole.  This  is  a  different  number,  developed  under 
a  different  philosophy  and  different  threat  conditions,  than  that  used  in  the  ASPJ  ORD. 


Eu.l . (I) 
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Because  the  1999 IDECM  program  restructure  included  a  schedule  adjustment  that 
precluded  use  of  the  AN/ALQ-214  on  the  first  several  deployments  oftheF/A-18E/F,  the 
F-l  8  community  opted  to  equip  the  aircraft  for  those  first  deployments  with  in-stock 
Navy  ASPJ  units  and  ALE-50  towed  decoys.  This  configuration,  referred  to  as  IDECM 
Block  1 ,  was  incorporated  into  the  restructured  IDECM  acquisition  plan  as  an  interim 
solution.  Because  the  ASPJ  on-board  transmitters  and  the  AN/ALQ-214  on-board 
transmitters  are  the  same  WRA  items,  the  Block  1  on-board  transmitters  were  also 
evaluated  against  the  requirements  of  the  On-Board  Jammer  ORD. 

We  do  not  concur  with  the  finding  that  the  Navy’s  logistical  support  requirement  may 
have  to  be  significantly  increased.  The  current  logistics  support  cost  requirements  are 
based  upon  the  ALQ-214  system  (including  on-board  transmitters)  as  it  is  intended  to  be 
fielded.  The  logistics  support  cost  is  based  upon  a  support  analysis  that  uses  the  predicted 
system  mean  time  between  failures,  component  predicted  mean  time  between  failures, 
aircraft  flight  hours,  and  operational  availability  to  determine  the  best  maintenance 
solution  at  the  lowest  cost.  These  factors  are  also  used  to  determine  the  sparing  levels 
required  for  the  system. 

We  do  not  concur  with  the  finding  that  it  is  unclear  whether  the  additional  protection 
provided  by  the  on-board  jamming  capability  justifies  the  investment  in  the  development, 
acquisition,  and  logistical  support.  The  addition  of  on-board  jammers  are  expected  to 
improve  survivability  under  three  circumstances:  (1)  if  the  threat  system  is  below  the 
radio  frequency  (RF)  coverage  of  the  decoy,  (2)  if  a  decoy  is  not  deployed  during  the 
engagement,  and  (3)  during  the  time  between  when  a  decoy  is  destroyed  by  a  missile  and 
the  next  decoy  is  fully  deployed  and  operating.  As  non-developmental  items,  the  on¬ 
board  transmitters  have  minimal  impact  in  terms  of  development,  acquisition  and  logistic 
support  costs. 

Department  of  the  Navy  Specific  Comments: 

Page  8,  paragraph  6,  Aircraft  Attrition.  We  concur  with  the  finding  that  reduction  in 
mission  reliability  produces  only  slight  changes  in  the  predicted  number  of  aircraft  losses 
For  the  Cost  and  Operational  Effectiveness  Analysis  (COEA),  calculations  were 
performed  to  assess  the  impact  of  EW  suite  reliability  on  aircraft  attrition.  Assuming  that 
the  aircraft  would  be  in  the  threat  envelope  for  10  minutes,  the  expected  attrition  for  1 10 
strike  sorties  (22  F/A-18E/Fs  in  the  threat  envelope  over  five  successive  strikes)  for  mean 
flight  hours  between  critical  failure  (MFHBCF)  of  1 0  to  250  hours  was  plotted.  In  this 
case,  a  critical  failure  meant  that  no  RF  energy  was  transmitted  by  the  system. 
Examination  of  the  resulting  curve  showed  that  at  about  30  hours  MFHBCF,  the  attrition 
curve  starts  to  turn  up  and  so  the  COEA  stated  "...no  significant  increase  in  attrition  as 
long  as  the  MFHBCF  was  greater  than  about  30  hours."  However,  even  at  10  hours,  the 
curve  is  not  terribly  steep.  In  fact,  with  these  same  assumptions,  varying  MFHBCF  from 
30  to  12  hours  (reliability  from  94%  to  86%)  changes  the  expected  attrition  for  those  1 10 
sorties  from  X  to  Y  (NOTE:  X  and  Y  are  classified  numbers  that  may  be  provided  upon 
contacting  the  IDECM  Program  Office).  Although  this  is  an  increase  of  7%,  in  terms  of 
aircraft  losses  over  the  1 10  sorties  the  actual  number  is  quite  small. 
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It  should  also  be  noted  that  the  addition  of  the  on-board  jammers  adds  an  alternate 
means  of  jamming  in  the  event  that  a  decoy  is  not  operational.  Although  the  reliability  of 
the  entire  EW  suite  (on-board  and  off-board)  may  be  less  than  what  was  recommended  in 
the  COEA,  a  failure  in  this  case  could  mean  that  the  system  would  be  capable  of 
transmitting  from  the  on-board  transmitters  but  not  the  decoy,  or  vice  versa.  Therefore, 
the  expected  attrition  in  the  event  of  a  failure  would  actually  be  less  than  the  previous 
estimate. 

Recommendations: 

We  recommend  that  the  Program  Manager,  Integrated  Defensive  Electronic 
Countermeasures,  prior  to  proceeding  with  Block  II  of  the  Program: 

1.  Determine  the  logistics  support  cost  to  maintain  the  on-board  jamming  capability 
based  on  the  results  of  the  Operational  Test  and  Evaluation. 

Department  of  the  Navy  Comments: 

Concur  with  the  intent  of  the  recommendation.  We  intend  to  review  the  logistic 
support  costs  in  the  normal  course  of  preparation  for  the  Milestone  III  decision. 

In  accordance  with  the  IDECM  Acquisition  Plan,  ASN  (RD&A)  approved  entry  into 
Limited  Rate  Initial  Production  for  the  AN/ALQ-214  system  in  November  2000. 
Operational  Test  and  Evaluation  will  begin  in  September  2001  to  support  a  Third 
Quarter,  FY02  Milestone  III  Decision.  All  appropriate  acquisition  data  and 
documentation  will  be  reviewed  in  preparation  for  this  milestone  decision. 

The  logistics  support  cost  is  based  upon  a  support  analysis  that  uses  the  predicted 
system  mean  time  between  failures,  component  predicted  mean  time  between  failures, 
aircraft  flight  hours,  and  operational  availability  to  determine  the  best  maintenance 
solution  at  the  lowest  cost.  These  factors  are  also  used  to  determine  the  sparing  levels 
required  for  the  system.  This  analysis  will  he  reviewed  and  updated  as  necessary  in 
preparation  for  the  milestone  decision. 

2.  Reassess  whether  the  on-board  jamming  capability  provides  an  amount  of  added 
protection  that  justifies  its  development,  acquisition  and  logistical  support  costs. 

Department  of  the  Navy  Comments: 

Do  not  concur.  This  issue  was  reassessed  in  the  Johns  Hopkins  University  Applied 
Physics  Laboratory  SECRET  Report,  VAS-99-022,  "The  Use  of  On-board  Transmitters 
with  the  Integrated  Defensive  Electronic  Countermeasures  System  (U),''  20  May  1999. 

We  do  not  believe  another  assessment  at  this  point  is  necessary.  All  appropriate 
acquisition  data  and  documentation  will  be  reviewed  in  preparation  for  the  Milestone  III 
decision,  scheduled  for  Third  Quarter,  FY02.  As  non-developmental  items,  the  on-board 
transmitters  have  minimal  impact  in  terms  of  development,  acquisition  and  logistic 
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support  costs.  The  very  low  development  cost  of  the  on-board  transmitter  can  not  be 
broken  out  from  other  system  development  costs  and  has,  for  the  most  part,  already  been 
paid. 

The  addition  of  on-board  jammers  is  expected  to  improve  survivability  under  three 
circumstances: 

1 .  The  first  occurs  if  the  threat  system  is  below  the  RF  coverage  of  the  decoy.  The 
Surface-to-Air  Missiles  (SAMs)  that  currently  operate  in  the  lower  RF  range  tend  to  be 
less  sophisticated.  Therefore,  they  can  generally  be  defeated  with  a  combination  standoff 
jamming  (EA-6Bs),  maneuvers,  and  chaff.  (The  latter  two  may  require  some  indication 
of  missile  launch.)  In  addition,  the  large  size  of  their  antenna  make  them  less  mobile  and 
therefore  better  targets  for  lethal  suppression.  However,  both  of  these  limitations  are 
being  mitigated  somewhat  by  upgrades  to  these  SAMs  and  innovative  ways  of  increasing 
their  mobility,  making  the  addition  of  on-board  transmitters  more  beneficial. 

2.  The  second  benefit  of  on-board  jammers  is  the  protection  that  they  provide 
during  the  time  when  there  is  no  decoy  deployed.  This  could  be  because  of  tactics;  e.g., 
in  low  threat  environments  the  aircraft  may  enable  their  on-board  system  once  airborne 
but  not  deploy  a  decoy  until  there  is  some  indication  of  hostility.  In  fact,  for  sustained 
operations  such  as  no-fly  zone  enforcement,  this  tactic  may  actually  save  money  because 
a  decoy  would  not  be  used  for  each  sortie.  Of  course,  the  risk  associated  with  this  tactic 
must  be  considered.  The  other  situation  where  the  on-board  system  is  operational 
without  a  decoy  is  if  all  the  decoys  have  been  expended.  For  the  tactics  modeled  in  the 
COEA,  it  was  unlikely  that  an  aircraft  would  be  engaged  after  it  lost  all  of  its  decoys. 
Although  the  COEA  accounted  for  decoys  destroyed  by  missiles,  it  did  not  account  for 
decoys  lost  because  of  bumoff.  At  the  time  of  the  COEA,  this  was  not  assessed  to  be  a 
significant  problem.  Since  that  time,  testing  has  shown  that  it  could  be  a  concern 
depending  on  the  tactics  employed.  Therefore,  the  likelihood  of  an  engagement  after  all 
of  the  decoys  have  been  expended  may  be  higher  than  assessed  in  the  COEA,  and  the 
benefit  of  the  on-board  system  in  this  case  may  be  significant. 

3.  The  on-board  jammer  may  also  provide  benefit  during  the  time  between  when  a 
missile  destroys  a  decoy  and  the  next  decoy  is  deployed  and  becomes  operational. 
Although  the  deployment  occurs  relatively  quickly,  there  may  be  some  period  of  reduced 
protection  while  it  is  moving  away  from  the  aircraft  if  the  missile  that  destroys  the  decoy 
is  the  first  missile  in  a  salvo.  In  this  case,  the  on-board  jammer  may  compensate  for 
some  of  that  reduced  protection. 

All  of  the  above  benefits  are  very  difficult  to  quantify  without  assumptions  and/or  data 
on  the  likelihood  of  various  events,  e.g.,  the  likelihood  that  a  decoy  will  bumoff  during 
some  portion  of  a  strike  mission.  In  addition,  recent  conflicts  have  highlighted  the  high 
political  costs  associated  with  the  loss  of  aircraft  and  pilots.  These  may  far  outweigh  the 
associated  economic  or  military  costs.  Therefore,  the  added  protection  provided  by  the 
on-board  system  may  provide  a  valuable  benefit  that  cannot  be  quantified. 
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